<!DOCTYPE html>
<html>
  <head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8">
  <meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=0" name="viewport">
  <meta name="description" content="刘清政">
  <meta name="keyword" content="hexo-theme">
  
    <link rel="shortcut icon" href="/css/images/logo.png">
  
  <title>
    
      1-Python中的GIL | Justin-刘清政的博客
    
  </title>
  <link href="//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet">
  <link href="//cdnjs.cloudflare.com/ajax/libs/nprogress/0.2.0/nprogress.min.css" rel="stylesheet">
  <link href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/tomorrow.min.css" rel="stylesheet">
  
<link rel="stylesheet" href="/css/style.css">

  
    
<link rel="stylesheet" href="/css/plugins/gitment.css">

  
  <script src="//cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
  <script src="//cdnjs.cloudflare.com/ajax/libs/geopattern/1.2.3/js/geopattern.min.js"></script>
  <script src="//cdnjs.cloudflare.com/ajax/libs/nprogress/0.2.0/nprogress.min.js"></script>
  
    
<script src="/js/qrious.js"></script>

  
  
    
<script src="/js/gitment.js"></script>

  
  

  
<meta name="generator" content="Hexo 4.2.0"></head>
<div class="wechat-share">
  <img src="/css/images/logo.png" />
</div>

  <body>
    <header class="header fixed-header">
  <div class="header-container">
    <a class="home-link" href="/">
      <div class="logo"></div>
      <span>Justin-刘清政的博客</span>
    </a>
    <ul class="right-list">
      
        <li class="list-item">
          
            <a href="/" class="item-link">主页</a>
          
        </li>
      
        <li class="list-item">
          
            <a href="/tags/" class="item-link">标签</a>
          
        </li>
      
        <li class="list-item">
          
            <a href="/archives/" class="item-link">归档</a>
          
        </li>
      
        <li class="list-item">
          
            <a href="/about/" class="item-link">关于我</a>
          
        </li>
      
    </ul>
    <div class="menu">
      <span class="icon-bar"></span>
      <span class="icon-bar"></span>
      <span class="icon-bar"></span>
    </div>
    <div class="menu-mask">
      <ul class="menu-list">
        
          <li class="menu-item">
            
              <a href="/" class="menu-link">主页</a>
            
          </li>
        
          <li class="menu-item">
            
              <a href="/tags/" class="menu-link">标签</a>
            
          </li>
        
          <li class="menu-item">
            
              <a href="/archives/" class="menu-link">归档</a>
            
          </li>
        
          <li class="menu-item">
            
              <a href="/about/" class="menu-link">关于我</a>
            
          </li>
        
      </ul>
    </div>
  </div>
</header>

    <div id="article-banner">
  <h2>1-Python中的GIL</h2>



  <p class="post-date">2020-03-25</p>
    <!-- 不蒜子统计 -->
    <span id="busuanzi_container_page_pv" style='display:none' class="">
        <i class="icon-smile icon"></i> 阅读数：<span id="busuanzi_value_page_pv"></span>次
    </span>
  <div class="arrow-down">
    <a href="javascript:;"></a>
  </div>
</div>
<main class="app-body flex-box">
  <!-- Article START -->
  <article class="post-article">
    <section class="markdown-content"><h1 id="一、到底什么是线程-什么是进程"><a href="#一、到底什么是线程-什么是进程" class="headerlink" title="一、到底什么是线程?什么是进程?"></a>一、到底什么是线程?什么是进程?</h1><p>Python自己没有这玩意，Python中调用的操作系统的线程和进程。</p>
<h1 id="二、Python多线程情况下："><a href="#二、Python多线程情况下：" class="headerlink" title="二、Python多线程情况下："></a>二、Python多线程情况下：</h1><p>计算密集型操作:效率低，Python内置的一个全局解释器锁，锁的作用就是保证同一时刻一个进程中只有一个线程可以被cpu调度，多线程无法利用多核优势，可以通过多进程方式解决，但是比较浪费资源。<br>IO操作:效率高</p>
<h1 id="三、Python多进程的情况下："><a href="#三、Python多进程的情况下：" class="headerlink" title="三、Python多进程的情况下："></a>三、Python多进程的情况下：</h1><p>计算密集型操作:效率高(浪费资源)，不得已而为之。<br>IO操作:效率高(浪费资源)</p>
<h1 id="四、为什么有这把GIL锁"><a href="#四、为什么有这把GIL锁" class="headerlink" title="四、为什么有这把GIL锁?"></a>四、为什么有这把GIL锁?</h1><p>Python语言的创始人在开发这门语言时，目的快速把语言开发出来，如果加上GIL锁(C语言加锁)，切换时按照100条字节指令来进行线程间的切换。</p>
<h1 id="五、Python中线程和进程-GIL锁"><a href="#五、Python中线程和进程-GIL锁" class="headerlink" title="五、Python中线程和进程(GIL锁)"></a>五、Python中线程和进程(GIL锁)</h1><p>GIL锁，全局解释器锁。用于限制一个进程中同一时刻只有一个线程被cpu调度。<br>扩展:默认GIL锁在执行100个cpu指令(过期时间)。<br>查看GIL切换的指令个数</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> sys</span><br><span class="line">v1 = sys。getcheckinterval()</span><br><span class="line">print(v1)</span><br></pre></td></tr></table></figure>

<h1 id="六、为什么要创建线程？"><a href="#六、为什么要创建线程？" class="headerlink" title="六、为什么要创建线程？"></a>六、为什么要创建线程？</h1><p>由于线程是cpu工作的最小单元，创建线程可以利用多核优势实现并行操作(Java/C#)。<br>注意：<strong>线程是为了工作。</strong></p>
<h1 id="七、为什么要创建进程？"><a href="#七、为什么要创建进程？" class="headerlink" title="七、为什么要创建进程？"></a>七、为什么要创建进程？</h1><p>进程和进程之间做数据隔离(Java/C#)。</p>
<p>注意：<strong>进程是为了提供环境让线程工作。</strong></p>
<h1 id="八、进程和线程的区别？"><a href="#八、进程和线程的区别？" class="headerlink" title="八、进程和线程的区别？"></a>八、进程和线程的区别？</h1><p>进程是资源分配的最小单位，线程是程序执行的最小单位。</p>
<p>进程有自己的独立地址空间，每启动一个进程，系统就会为它分配地址空间，建立数据表来维护代码段，堆栈段和数据段，这种操作非常昂贵。而线程是共享进程中的数据的，使用相同的地址空间，因此CPU切换一个线程的花费远比进程小很多，同时创建一个线程的开销比进程要小很多。</p>
<p>线程之间的通信更方便，同一进程下的线程共享全局变量，静态变量，而进程之间的通信需要以通信的方式（IPC)进行。不过如何处理好同步与互斥是编写多线程程序的难点。</p>
<p>但是多进程程序更健壮，多线程程序只要有一个线程死掉，整个进程也死掉了，而一个进程死掉并不会对另外一个进程造成影响，因为进程有自己独立的地址空间。</p>
<p>也正是由于GIL锁的原因：IO密集型操作可以使用多线程，计算密集型可以使用多进程。</p>
<h1 id="九、线程创建的越多越好吗"><a href="#九、线程创建的越多越好吗" class="headerlink" title="九、线程创建的越多越好吗?"></a>九、线程创建的越多越好吗?</h1><p>不好。线程之间进行切换时，要做上下文管理。</p>
<h1 id="十、生产者消费者模型解决了什么问题"><a href="#十、生产者消费者模型解决了什么问题" class="headerlink" title="十、生产者消费者模型解决了什么问题?"></a>十、生产者消费者模型解决了什么问题?</h1><p>不用一直等待的问题。</p>
<h1 id="十一、Lock和RLock的区别？"><a href="#十一、Lock和RLock的区别？" class="headerlink" title="十一、Lock和RLock的区别？"></a>十一、Lock和RLock的区别？</h1><p>RLock可以多次加锁。</p>
<h1 id="十二、进程和线程以及协程的区别？"><a href="#十二、进程和线程以及协程的区别？" class="headerlink" title="十二、进程和线程以及协程的区别？"></a>十二、进程和线程以及协程的区别？</h1><p>进程是cpu资源分配的最小单元，一个进程中可以有多个线程。</p>
<p>线程是cpu计算的最小单元。</p>
<p>对于Python来说他的进程和线程和其他语言有差异，是有GIL锁。</p>
<p>GIL锁保证一个进程中同一时刻只有一个线程被cpu调度。</p>
<p>注意：<strong>IO密集型操作可以使用多线程，计算密集型可以使用多进程。</strong></p>
<p>协程，是由程序员创造出来的一个不是真实存在的东西。</p>
<p>协程，是微线程，对一个线程进程分片，使得线程在代码块之间进行来回切换执行，而不是在原来逐行执行。</p>
<h1 id="十三、IO多路复用作用"><a href="#十三、IO多路复用作用" class="headerlink" title="十三、IO多路复用作用?"></a>十三、IO多路复用作用?</h1><p>检测多个socket是否已经发生变化（是否已经连接成功/是否已经获取数据）(可读/可写)IO多路复用作用？</p>
<p>检测多个socket是否发生变化。</p>
<p>操作系统检测socket是否发生变化，有三种模式：</p>
<ul>
<li>select：最多1024个socket；循环去检测。</li>
<li>poll：不限制监听socket个数；循环去检测（水平触发）。</li>
<li>epoll：不限制监听socket个数；回调方式（边缘触发）。</li>
</ul>
<p>Python模块：</p>
<ul>
<li>select.select</li>
<li>select.epoll</li>
</ul>
<h1 id="十四、socket默认是否是阻塞的-阻塞体现在哪里"><a href="#十四、socket默认是否是阻塞的-阻塞体现在哪里" class="headerlink" title="十四、socket默认是否是阻塞的?阻塞体现在哪里?"></a>十四、socket默认是否是阻塞的?阻塞体现在哪里?</h1><p>默认是阻塞，填在等待消息和连接</p>
<h1 id="十五、什么是异步非阻塞"><a href="#十五、什么是异步非阻塞" class="headerlink" title="十五、什么是异步非阻塞?"></a>十五、什么是异步非阻塞?</h1><p>非阻塞，不等待。比如创建socket对某个地址进行connect、获取接收数据recv时默认都会等待（连接成功或接收到数据），才执行后续操作。<br>如果设置setblocking(False)，以上两个过程就不再等待，但是会报BlockingIOError的错误，只要捕获即可。</p>
<p>异步，通知，执行完成之后自动执行回调函数或自动执行某些操作（通知）。比如做爬虫中向某个地址baidu。com发送请求，当请求执行完成之后自执行回调函数。</p>
<h1 id="十六、什么是同步阻塞？"><a href="#十六、什么是同步阻塞？" class="headerlink" title="十六、什么是同步阻塞？"></a>十六、什么是同步阻塞？</h1><ul>
<li>阻塞：等</li>
<li>同步：按照顺序逐步执行</li>
</ul>
<h1 id="十七、什么是协程？"><a href="#十七、什么是协程？" class="headerlink" title="十七、什么是协程？"></a>十七、什么是协程？</h1><p>协程也可以称为“微线程”，就是开发者控制线程执行流程，控制先执行某段代码然后再切换到另外函执行代码…来回切换。</p>
<h1 id="十八、协程可以提高并发吗？"><a href="#十八、协程可以提高并发吗？" class="headerlink" title="十八、协程可以提高并发吗？"></a>十八、协程可以提高并发吗？</h1><p>协程自己本身无法实现并发（甚至性能会降低）。<br>协程+IO切换性能提高。</p>
<h1 id="十九、提高并发方案："><a href="#十九、提高并发方案：" class="headerlink" title="十九、提高并发方案："></a>十九、提高并发方案：</h1><ul>
<li>多进程</li>
<li>多线程</li>
<li>单线程提供并发</li>
</ul>
<h1 id="二十、单线程提供并发"><a href="#二十、单线程提供并发" class="headerlink" title="二十、单线程提供并发"></a>二十、单线程提供并发</h1><h2 id="20-1-基于IO多路复用-socket非阻塞"><a href="#20-1-基于IO多路复用-socket非阻塞" class="headerlink" title="20.1 基于IO多路复用+socket非阻塞"></a>20.1 基于IO多路复用+socket非阻塞</h2><p>实现并发请求(一个线程100个请求)</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> socket</span><br><span class="line"><span class="comment"># 创建socket</span></span><br><span class="line">client = socket.socket()</span><br><span class="line"><span class="comment"># 将原来阻塞的位置变成非阻塞（报错）</span></span><br><span class="line">client.setblocking(<span class="literal">False</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 百度创建连接: 阻塞</span></span><br><span class="line"><span class="keyword">try</span>:</span><br><span class="line">    <span class="comment"># 执行了但报错了</span></span><br><span class="line">    client.connect((<span class="string">'www.baidu.com'</span>,<span class="number">80</span>))</span><br><span class="line"><span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">    <span class="keyword">pass</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 检测到已经连接成功</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 问百度我要什么？</span></span><br><span class="line">client.sendall(<span class="string">b'GET /s?wd=alex HTTP/1.0\r\nhost:www.baidu.com\r\n\r\n'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 我等着接收百度给我的回复</span></span><br><span class="line">chunk_list = []</span><br><span class="line"><span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line">    <span class="comment"># 将原来阻塞的位置变成非阻塞（报错）</span></span><br><span class="line">    chunk = client.recv(<span class="number">8096</span>) </span><br><span class="line">    <span class="keyword">if</span> <span class="keyword">not</span> chunk:</span><br><span class="line">        <span class="keyword">break</span></span><br><span class="line">    chunk_list.append(chunk)</span><br><span class="line"></span><br><span class="line">body = <span class="string">b''</span>.join(chunk_list)</span><br><span class="line">print(body.decode(<span class="string">'utf-8'</span>))</span><br></pre></td></tr></table></figure>

<h2 id="20-2-协程-IO切换：gevent"><a href="#20-2-协程-IO切换：gevent" class="headerlink" title="20.2 协程+IO切换：gevent"></a>20.2 协程+IO切换：gevent</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> gevent <span class="keyword">import</span> monkey</span><br><span class="line"><span class="comment"># 以后代码中遇到IO都会自动执行greenlet的switch进行切换</span></span><br><span class="line">monkey.patch_all() </span><br><span class="line"><span class="keyword">import</span> requests</span><br><span class="line"><span class="keyword">import</span> gevent</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">get_page1</span><span class="params">(url)</span>:</span></span><br><span class="line">    ret = requests.get(url)</span><br><span class="line">    print(url,ret.content)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">get_page2</span><span class="params">(url)</span>:</span></span><br><span class="line">    ret = requests.get(url)</span><br><span class="line">    print(url,ret.content)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">get_page3</span><span class="params">(url)</span>:</span></span><br><span class="line">    ret = requests.get(url)</span><br><span class="line">    print(url,ret.content)</span><br><span class="line"></span><br><span class="line">gevent.joinall([</span><br><span class="line">    gevent.spawn(get_page1, <span class="string">'https://www.python.org/'</span>), <span class="comment"># 协程1</span></span><br><span class="line">    gevent.spawn(get_page2, <span class="string">'https://www.yahoo.com/'</span>),  <span class="comment"># 协程2</span></span><br><span class="line">    gevent.spawn(get_page3, <span class="string">'https://github.com/'</span>),     <span class="comment"># 协程3</span></span><br><span class="line">])</span><br></pre></td></tr></table></figure>

<h2 id="20-3-基于事件循环的异步"><a href="#20-3-基于事件循环的异步" class="headerlink" title="20.3 基于事件循环的异步"></a>20.3 基于事件循环的异步</h2><p>执行完某个人物后自动调用我给他的函数，非阻塞</p>
<h3 id="20-3-1-方法一"><a href="#20-3-1-方法一" class="headerlink" title="20.3.1 方法一"></a>20.3.1 方法一</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> socket</span><br><span class="line"><span class="keyword">import</span> select</span><br><span class="line"></span><br><span class="line"><span class="comment"># 百度创建连接:非阻塞</span></span><br><span class="line">client1 = socket.socket()</span><br><span class="line">client1.setblocking(<span class="literal">False</span>)</span><br><span class="line"><span class="keyword">try</span>:</span><br><span class="line">    client1.connect((<span class="string">'www.baidu.com'</span>, <span class="number">80</span>))</span><br><span class="line"><span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">    <span class="keyword">pass</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 搜狗创建连接:非阻塞</span></span><br><span class="line">client2 = socket.socket()</span><br><span class="line">client2.setblocking(<span class="literal">False</span>)</span><br><span class="line"><span class="keyword">try</span>:</span><br><span class="line">    client2.connect((<span class="string">'www.sogou.com'</span>, <span class="number">80</span>))</span><br><span class="line"><span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">    <span class="keyword">pass</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># GitHub创建连接:非阻塞</span></span><br><span class="line">client3 = socket.socket()</span><br><span class="line">client3.setblocking(<span class="literal">False</span>)</span><br><span class="line"><span class="keyword">try</span>:</span><br><span class="line">    client3.connect((<span class="string">'www.github.com'</span>, <span class="number">80</span>))</span><br><span class="line"><span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">    <span class="keyword">pass</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 创建socket列表:socket_list</span></span><br><span class="line">socket_list = [client1, client2, client3]</span><br><span class="line"><span class="comment"># 创建connect列表:conn_list</span></span><br><span class="line">conn_list = [client1, client2, client3]</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line">    rlist, wlist, elist = select.select(socket_list, conn_list, [], <span class="number">0.005</span>)</span><br><span class="line">    <span class="comment"># rlist中表示已近获取数据的socket对象</span></span><br><span class="line">    <span class="comment"># wlist中表示已经连接成功的socket对象</span></span><br><span class="line">    <span class="comment"># elist中表示出现错误的socket对象</span></span><br><span class="line">    <span class="keyword">for</span> sk <span class="keyword">in</span> wlist:</span><br><span class="line">        <span class="keyword">if</span> sk == client1:</span><br><span class="line">            sk.sendall(<span class="string">b'GET /s?wd=alex HTTP/1.0\r\nhost:www.baidu.com\r\n\r\n'</span>)</span><br><span class="line">        <span class="keyword">elif</span> sk == client2:</span><br><span class="line">            sk.sendall(<span class="string">b'GET /web?query=fdf HTTP/1.0\r\nhost:www.sogou.com\r\n\r\n'</span>)</span><br><span class="line">        <span class="keyword">else</span>:</span><br><span class="line">            sk.sendall(<span class="string">b'GET /s?wd=alex HTTP/1.0\r\nhost:www.oldboyedu.com\r\n\r\n'</span>)</span><br><span class="line">        conn_list.remove(sk)</span><br><span class="line">    <span class="keyword">for</span> sk <span class="keyword">in</span> rlist:</span><br><span class="line">        chunk_list = []</span><br><span class="line">        <span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line">            <span class="keyword">try</span>:</span><br><span class="line">                chunk = sk.recv(<span class="number">8096</span>)</span><br><span class="line">                <span class="keyword">if</span> <span class="keyword">not</span> chunk:</span><br><span class="line">                    <span class="keyword">break</span></span><br><span class="line">                chunk_list.append(chunk)</span><br><span class="line">            <span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">                <span class="keyword">break</span></span><br><span class="line">        body = <span class="string">b''</span>.join(chunk_list)</span><br><span class="line">        <span class="comment"># print(body.decode('utf-8'))</span></span><br><span class="line">        print(<span class="string">'------------&gt;'</span>, body)</span><br><span class="line">        sk.close()</span><br><span class="line">        socket_list.remove(sk)</span><br><span class="line">    <span class="keyword">if</span> <span class="keyword">not</span> socket_list:</span><br><span class="line">        <span class="keyword">break</span></span><br></pre></td></tr></table></figure>

<h3 id="20-3-2-方法二"><a href="#20-3-2-方法二" class="headerlink" title="20.3.2 方法二"></a>20.3.2 方法二</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> socket</span><br><span class="line"><span class="keyword">import</span> select</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Req</span><span class="params">(object)</span>:</span></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">__init__</span><span class="params">(self,sk,func)</span>:</span></span><br><span class="line">        self.sock = sk</span><br><span class="line">        self.func = func</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">fileno</span><span class="params">(self)</span>:</span></span><br><span class="line">        <span class="keyword">return</span> self.sock.fileno()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Nb</span><span class="params">(object)</span>:</span></span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">__init__</span><span class="params">(self)</span>:</span></span><br><span class="line">        self.conn_list = []</span><br><span class="line">        self.socket_list = []</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">add</span><span class="params">(self,url,func)</span>:</span></span><br><span class="line">        <span class="comment"># 创建socket客户端</span></span><br><span class="line">        client = socket.socket()</span><br><span class="line">        <span class="comment"># 非阻塞</span></span><br><span class="line">        client.setblocking(<span class="literal">False</span>)</span><br><span class="line">        <span class="keyword">try</span>:</span><br><span class="line">            <span class="comment"># 创建连接</span></span><br><span class="line">            client.connect((url, <span class="number">80</span>))</span><br><span class="line">            <span class="comment"># 异常处理</span></span><br><span class="line">        <span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">            <span class="keyword">pass</span></span><br><span class="line">        obj = Req(client,func)</span><br><span class="line">        <span class="comment"># 连接列表</span></span><br><span class="line">        self.conn_list.append(obj)</span><br><span class="line">        <span class="comment"># socket列表</span></span><br><span class="line">        self.socket_list.append(obj)</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">run</span><span class="params">(self)</span>:</span></span><br><span class="line"></span><br><span class="line">        <span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line">            rlist,wlist,elist = select.select(self.socket_list,self.conn_list,[],<span class="number">0.005</span>)</span><br><span class="line">            <span class="comment"># wlist中表示已经连接成功的req对象</span></span><br><span class="line">            <span class="keyword">for</span> sk <span class="keyword">in</span> wlist:</span><br><span class="line">                <span class="comment"># 发生变换的req对象</span></span><br><span class="line">                sk.sock.sendall(<span class="string">b'GET /s?wd=alex HTTP/1.0\r\nhost:www.baidu.com\r\n\r\n'</span>)</span><br><span class="line">                self.conn_list.remove(sk)</span><br><span class="line">            <span class="keyword">for</span> sk <span class="keyword">in</span> rlist:</span><br><span class="line">                chunk_list = []</span><br><span class="line">                <span class="keyword">while</span> <span class="literal">True</span>:</span><br><span class="line">                    <span class="keyword">try</span>:</span><br><span class="line">                        chunk = sk.sock.recv(<span class="number">8096</span>)</span><br><span class="line">                        <span class="keyword">if</span> <span class="keyword">not</span> chunk:</span><br><span class="line">                            <span class="keyword">break</span></span><br><span class="line">                        chunk_list.append(chunk)</span><br><span class="line">                    <span class="keyword">except</span> BlockingIOError <span class="keyword">as</span> e:</span><br><span class="line">                        <span class="keyword">break</span></span><br><span class="line">                body = <span class="string">b''</span>.join(chunk_list)</span><br><span class="line">                <span class="comment"># print(body.decode('utf-8'))</span></span><br><span class="line">                sk.func(body)</span><br><span class="line">                sk.sock.close()</span><br><span class="line">                self.socket_list.remove(sk)</span><br><span class="line">            <span class="keyword">if</span> <span class="keyword">not</span> self.socket_list:</span><br><span class="line">                <span class="keyword">break</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">baidu_repsonse</span><span class="params">(body)</span>:</span></span><br><span class="line">    print(<span class="string">'百度下载结果：'</span>,body)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">sogou_repsonse</span><span class="params">(body)</span>:</span></span><br><span class="line">    print(<span class="string">'搜狗下载结果：'</span>, body)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">github_repsonse</span><span class="params">(body)</span>:</span></span><br><span class="line">    print(<span class="string">'GITHUB下载结果：'</span>, body)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">t1 = Nb()</span><br><span class="line">t1.add(<span class="string">'www.baidu.com'</span>,baidu_repsonse)</span><br><span class="line">t1.add(<span class="string">'www.sogou.com'</span>,sogou_repsonse)</span><br><span class="line">t1.add(<span class="string">'www.github.com'</span>,oldboyedu_repsonse)</span><br><span class="line">t1.run()</span><br></pre></td></tr></table></figure>

<h2 id="20-4-基于事件循环的异步非阻塞框架"><a href="#20-4-基于事件循环的异步非阻塞框架" class="headerlink" title="20.4 基于事件循环的异步非阻塞框架"></a>20.4 基于事件循环的异步非阻塞框架</h2><p>如Twisted框架，scrapy框架(单线程完成并发)</p>
</section>
    <!-- Tags START -->
    
      <div class="tags">
        <span>Tags:</span>
        
  <a href="/tags#Python" >
    <span class="tag-code">Python</span>
  </a>

      </div>
    
    <!-- Tags END -->
    <!-- NAV START -->
    
  <div class="nav-container">
    <!-- reverse left and right to put prev and next in a more logic postition -->
    
      <a class="nav-left" href="/python/Python%E5%B9%B6%E5%8F%91%E7%BC%96%E7%A8%8B/11-%E8%BF%9B%E7%A8%8B%E6%B1%A0(multiprocess.Pool)/">
        <span class="nav-arrow">← </span>
        
          1-Python中的GIL
        
      </a>
    
    
      <a class="nav-right" href="/python/Python%E5%B9%B6%E5%8F%91%E7%BC%96%E7%A8%8B/19-%E9%94%81-%E4%BF%A1%E5%8F%B7%E9%87%8F-Event%E4%BA%8B%E4%BB%B6/">
        
          1-Python中的GIL
        
        <span class="nav-arrow"> →</span>
      </a>
    
  </div>

    <!-- NAV END -->
    <!-- 打赏 START -->
    
      <div class="money-like">
        <div class="reward-btn">
          赏
          <span class="money-code">
            <span class="alipay-code">
              <div class="code-image"></div>
              <b>使用支付宝打赏</b>
            </span>
            <span class="wechat-code">
              <div class="code-image"></div>
              <b>使用微信打赏</b>
            </span>
          </span>
        </div>
        <p class="notice">点击上方按钮,请我喝杯咖啡！</p>
      </div>
    
    <!-- 打赏 END -->
    <!-- 二维码 START -->
    
      <div class="qrcode">
        <canvas id="share-qrcode"></canvas>
        <p class="notice">扫描二维码，分享此文章</p>
      </div>
    
    <!-- 二维码 END -->
    
      <!-- Gitment START -->
      <div id="comments"></div>
      <!-- Gitment END -->
    
  </article>
  <!-- Article END -->
  <!-- Catalog START -->
  
    <aside class="catalog-container">
  <div class="toc-main">
  <!-- 不蒜子统计 -->
    <strong class="toc-title">目录</strong>
    
      <ol class="toc-nav"><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#一、到底什么是线程-什么是进程"><span class="toc-nav-text">一、到底什么是线程?什么是进程?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#二、Python多线程情况下："><span class="toc-nav-text">二、Python多线程情况下：</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#三、Python多进程的情况下："><span class="toc-nav-text">三、Python多进程的情况下：</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#四、为什么有这把GIL锁"><span class="toc-nav-text">四、为什么有这把GIL锁?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#五、Python中线程和进程-GIL锁"><span class="toc-nav-text">五、Python中线程和进程(GIL锁)</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#六、为什么要创建线程？"><span class="toc-nav-text">六、为什么要创建线程？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#七、为什么要创建进程？"><span class="toc-nav-text">七、为什么要创建进程？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#八、进程和线程的区别？"><span class="toc-nav-text">八、进程和线程的区别？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#九、线程创建的越多越好吗"><span class="toc-nav-text">九、线程创建的越多越好吗?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十、生产者消费者模型解决了什么问题"><span class="toc-nav-text">十、生产者消费者模型解决了什么问题?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十一、Lock和RLock的区别？"><span class="toc-nav-text">十一、Lock和RLock的区别？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十二、进程和线程以及协程的区别？"><span class="toc-nav-text">十二、进程和线程以及协程的区别？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十三、IO多路复用作用"><span class="toc-nav-text">十三、IO多路复用作用?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十四、socket默认是否是阻塞的-阻塞体现在哪里"><span class="toc-nav-text">十四、socket默认是否是阻塞的?阻塞体现在哪里?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十五、什么是异步非阻塞"><span class="toc-nav-text">十五、什么是异步非阻塞?</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十六、什么是同步阻塞？"><span class="toc-nav-text">十六、什么是同步阻塞？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十七、什么是协程？"><span class="toc-nav-text">十七、什么是协程？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十八、协程可以提高并发吗？"><span class="toc-nav-text">十八、协程可以提高并发吗？</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#十九、提高并发方案："><span class="toc-nav-text">十九、提高并发方案：</span></a></li><li class="toc-nav-item toc-nav-level-1"><a class="toc-nav-link" href="#二十、单线程提供并发"><span class="toc-nav-text">二十、单线程提供并发</span></a><ol class="toc-nav-child"><li class="toc-nav-item toc-nav-level-2"><a class="toc-nav-link" href="#20-1-基于IO多路复用-socket非阻塞"><span class="toc-nav-text">20.1 基于IO多路复用+socket非阻塞</span></a></li><li class="toc-nav-item toc-nav-level-2"><a class="toc-nav-link" href="#20-2-协程-IO切换：gevent"><span class="toc-nav-text">20.2 协程+IO切换：gevent</span></a></li><li class="toc-nav-item toc-nav-level-2"><a class="toc-nav-link" href="#20-3-基于事件循环的异步"><span class="toc-nav-text">20.3 基于事件循环的异步</span></a><ol class="toc-nav-child"><li class="toc-nav-item toc-nav-level-3"><a class="toc-nav-link" href="#20-3-1-方法一"><span class="toc-nav-text">20.3.1 方法一</span></a></li><li class="toc-nav-item toc-nav-level-3"><a class="toc-nav-link" href="#20-3-2-方法二"><span class="toc-nav-text">20.3.2 方法二</span></a></li></ol></li><li class="toc-nav-item toc-nav-level-2"><a class="toc-nav-link" href="#20-4-基于事件循环的异步非阻塞框架"><span class="toc-nav-text">20.4 基于事件循环的异步非阻塞框架</span></a></li></ol></li></ol>
    
  </div>
</aside>
  
  <!-- Catalog END -->
</main>

<script>
  (function () {
    var url = 'http://www.liuqingzheng.top/python/Python并发编程/26-并发编程小结/';
    var banner = ''
    if (banner !== '' && banner !== 'undefined' && banner !== 'null') {
      $('#article-banner').css({
        'background-image': 'url(' + banner + ')'
      })
    } else {
      $('#article-banner').geopattern(url)
    }
    $('.header').removeClass('fixed-header')

    // error image
    $(".markdown-content img").on('error', function() {
      $(this).attr('src', 'http://file.muyutech.com/error-img.png')
      $(this).css({
        'cursor': 'default'
      })
    })

    // zoom image
    $(".markdown-content img").on('click', function() {
      var src = $(this).attr('src')
      if (src !== 'http://file.muyutech.com/error-img.png') {
        var imageW = $(this).width()
        var imageH = $(this).height()

        var zoom = ($(window).width() * 0.95 / imageW).toFixed(2)
        zoom = zoom < 1 ? 1 : zoom
        zoom = zoom > 2 ? 2 : zoom
        var transY = (($(window).height() - imageH) / 2).toFixed(2)

        $('body').append('<div class="image-view-wrap"><div class="image-view-inner"><img src="'+ src +'" /></div></div>')
        $('.image-view-wrap').addClass('wrap-active')
        $('.image-view-wrap img').css({
          'width': `${imageW}`,
          'transform': `translate3d(0, ${transY}px, 0) scale3d(${zoom}, ${zoom}, 1)`
        })
        $('html').css('overflow', 'hidden')

        $('.image-view-wrap').on('click', function() {
          $(this).remove()
          $('html').attr('style', '')
        })
      }
    })
  })();
</script>


  <script>
    var qr = new QRious({
      element: document.getElementById('share-qrcode'),
      value: document.location.href
    });
  </script>



  <script>
    var gitmentConfig = "liuqingzheng";
    if (gitmentConfig !== 'undefined') {
      var gitment = new Gitment({
        id: "1-Python中的GIL",
        owner: "liuqingzheng",
        repo: "FuckBlog",
        oauth: {
          client_id: "32a4076431cf39d0ecea",
          client_secret: "94484bd79b3346a949acb2fda3c8a76ce16990c6"
        },
        theme: {
          render(state, instance) {
            const container = document.createElement('div')
            container.lang = "en-US"
            container.className = 'gitment-container gitment-root-container'
            container.appendChild(instance.renderHeader(state, instance))
            container.appendChild(instance.renderEditor(state, instance))
            container.appendChild(instance.renderComments(state, instance))
            container.appendChild(instance.renderFooter(state, instance))
            return container;
          }
        }
      })
      gitment.render(document.getElementById('comments'))
    }
  </script>




    <div class="scroll-top">
  <span class="arrow-icon"></span>
</div>
    <footer class="app-footer">
<!-- 不蒜子统计 -->
<span id="busuanzi_container_site_pv">
     本站总访问量<span id="busuanzi_value_site_pv"></span>次
</span>
<span class="post-meta-divider">|</span>
<span id="busuanzi_container_site_uv" style='display:none'>
     本站访客数<span id="busuanzi_value_site_uv"></span>人
</span>
<script async src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script>



  <p class="copyright">
    &copy; 2021 | Proudly powered by <a href="https://www.cnblogs.com/xiaoyuanqujing" target="_blank">小猿取经</a>
    <br>
    Theme by <a href="https://www.cnblogs.com/xiaoyuanqujing" target="_blank" rel="noopener">小猿取经</a>
  </p>
</footer>

<script>
  function async(u, c) {
    var d = document, t = 'script',
      o = d.createElement(t),
      s = d.getElementsByTagName(t)[0];
    o.src = u;
    if (c) { o.addEventListener('load', function (e) { c(null, e); }, false); }
    s.parentNode.insertBefore(o, s);
  }
</script>
<script>
  async("//cdnjs.cloudflare.com/ajax/libs/fastclick/1.0.6/fastclick.min.js", function(){
    FastClick.attach(document.body);
  })
</script>

<script>
  var hasLine = 'true';
  async("//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js", function(){
    $('figure pre').each(function(i, block) {
      var figure = $(this).parents('figure');
      if (hasLine === 'false') {
        figure.find('.gutter').hide();
      }
      var lang = figure.attr('class').split(' ')[1] || 'code';
      var codeHtml = $(this).html();
      var codeTag = document.createElement('code');
      codeTag.className = lang;
      codeTag.innerHTML = codeHtml;
      $(this).attr('class', '').empty().html(codeTag);
      figure.attr('data-lang', lang.toUpperCase());
      hljs.highlightBlock(block);
    });
  })
</script>





<!-- Baidu Tongji -->

<script>
    var _baId = 'c5fd96eee1193585be191f318c3fa725';
    // Originial
    var _hmt = _hmt || [];
    (function() {
      var hm = document.createElement("script");
      hm.src = "//hm.baidu.com/hm.js?" + _baId;
      var s = document.getElementsByTagName("script")[0];
      s.parentNode.insertBefore(hm, s);
    })();
</script>


<script src="/js/script.js"></script>


<script src="/js/search.js"></script>


<script src="/js/load.js"></script>



  <span class="local-search local-search-google local-search-plugin" style="right: 50px;top: 70px;;position:absolute;z-index:2;">
      <input type="search" placeholder="站内搜索" id="local-search-input" class="local-search-input-cls" style="">
      <div id="local-search-result" class="local-search-result-cls"></div>
  </span>


  </body>
</html>